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Non-linear picture processing EPO - DG 1 

1 7. 07. 2002 



The present invention relates to picture signal processing and, in particular, but 
not exclusively, to adaptive improvement of picture quality. 



Although high quality display devices are currently available for displaying, 
for example, video and computer-generated images, limitations nevertheless still exists 
particularly with digital displays such as LCD display panels and monitors. There is therefore 
recognition that there is further room for improvement in image quality and it is appreciated 
that a noticeable improvement to image quality could be brought about by achieving 
increases in color saturation. However, display devices commonly exhibit a maximum 
acceptabl e saturation v alue and it has been found to be the case that, while som e areas of a 
displayed knage, with particular colors, ncught well be at, or near, their maximum saturation 
value, other areas with different colors might still be someway below such maximimi value. 

Yet further, and particularly with current digital LCD monitors, relatively few 
picture adjustments are available to the user. Most frequently picture contrast and brightness 
can be adjusted, while saturation, hue and color temperature are less frequently available. In 
such digital displays, each pixel is driven by a triplet of values wherein each one represents 
the intensity of one of the three colors red, green and blue. However, due to the finite nimiber 
of bits serving to represent each color value, the possible driving values are likewise lixnited. 
For example, in a 24-bit LCD display panel, each color can be driven with any value included 
within a range of 0 to 255. Thus, if there is an attempt, either through some form of picture 
processing, or indeed user control, to increase the output over the maximum allowed value, a 
clipping effect will therefore result causing all the driving levels over the threshold to be 
represented by the same value, i.e. the maximirai saturation valuei. Thus, some picture detail 
quite clearly present in the input picture signal will therefore be lost through the 
amalgamations of the saturation levels caused by such clipping effect. 

It is an object of the invention to enhance images. The invention is defined by 
the independent claims. The dependent claims define advantageous embodiments. 
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According to one aspect of tibe present invention, there is provided a method 
of non-linear processing of input picture signals characterized in that the non-linear 
processing is responsive to hue values of the input picture signals. 

It is a particular advantage that the processing can serve to limit, or indeed 
5 avoid, any clipping of the output signal. This can be achieved by increasing the saturation up 
to a maximum level, at which clipping does not yet take place. Use is made of amongst 
others the hue values to determine this maximum level 

The invention has the advantage that, for example, color saturation of the 
output picture signals can thereby be controlled in a manner avoiding loss of picture-detail, 
10 since the color saturation is increased more for picture areas showing low saturation levels 
tiian for picture areas showing high saturation levels. The cUpping effect discussed above is 
therefore advantageously avoided. 

The set of mput parameter values preferably comprises the saturation values, 
as an increase of the saturation considerably enhances the image to be displayed. 

jjj jyteroatiyely.ad^tations.ofjQifLhrightnm vnlues can he applifid.3y.suitably 

combining adaptations of saturation values as weU as bri^tness values, there wiU be no 
undesired discoloration of the ima^. 

A simple method of non-linear processing is raising the parameter values to a 
certain power. By selecting a power depending on the hue values the non-Unear processmg 
20 takes into account that the output picture signals show no (or a negligible) clipping effect 

The non-linear processing can further be refined (in the sense of; for example, 
maximizing the saturation, while avoiding or minimizing cUpping), by taking into account 
the maximum saturation value Smax during the non-linear processing of the saturation values. 

It is advantageous if the maximum saturation value Smax is dependent on the 
25 hue values or the output parameter values corresponding to the brightness (preferably 
dependu^ on both h\ie vahies as well as ou^ut brightness values). 

The powers used in the non-linear processing can also be made dependent on 
the content of the image, represented by a histogram of one or more sets of input parameter 
values. This is a further refinement to adapt the non-linear processing in such a way that, for 
30 example, saturation is further hicreased, while clipping effects are fiirther minimized. 

In case of input picture signals represrating real-thne video information, the 
video information can be considered as a sequence of images. Each image has its own 
histogram. The powers can be adapted directly depending on the content of the histogram, or 
alternatively, a more gradual change can be applied by adapting the powers based on an 
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averaging over several images (low-pass filtering) of the changes of the powers. In this way 
irregularities are avoided, which would arise from sudden large changes of the powers from 
image to image. 

Thus, it should be appreciated that, with the present invention, the saturation 
5 and/or the brightness of each pixel within a display can be modified in a non-linear manner 
that serves to introduce significant enhancements where the input picture signals are having 
low satumtion values, less significant enhancements where the saturation values of the input 
signals are close to their maximum limits, and no changes \7vhere the input signals ^re having 
maximum saturation values already. 

10 

These and other aspects of the invention will be apparent from and elucidated 
with reference to the drawings in which: 

Fig. 1 is a basic block diagram of a system embodying the general concept of 
the present invention; 

15 Fig. 2 IS a scbematic-blockdiagram sTinwirigtViR gygtftmnf Pig i irpf^irrt^^r 

detail; 

Fig. 3 is a schematic block diagram illustrating one particular implementation 
of the non-linear saturation processing block of Fig. 2; 

Fig. 4 is a schematic block diagram of non-Unear luminance processing block 

20 of Fig. 2; 

Fig. Sa illustrates a graphical representation between niayimun> saturation and 
brightness for a given hue; 

Figs. 5b to 5d illustrate suitable values for intermediate parameters. 
Figs. 6a and 6b comprise schematic block diagrams of a hardware 
25 implementation of the saturation bound evaluation block of Fig. 2; and 
Fig,7 illustrates suitable values for the power y ^ . 
Fig. 8 illustrates suitable values for the other power y y . 

Turning first to Fig. 1, there is provided a generic block diagram illustrating a 
30 system 10 arranged for driving a display device in accordance with an embodiment of the 
present invention and which includes a non-linear processing engine 12 arranged to receive 
red, green and input pictures signals 14 and, once processed, to outpxit red, green and blue 
output picture signals 16 for a display device such as an LCD panel. The operation of flie 
non-linear processing engme 12 is controlled by means of a microcontroller 1 8 which is itself 
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responsive to signals from a histogram evaluator 20 which samples the red, green and blue 
mput picture signals 14 or parameters derived from these signals (not shown). In this 
illustrated example, the histogram evaluator 20 works on bri^tness, saturation and hue 
values of the input picture signals as will be illustrated further with reference to Fig. 2. Of 
5 course, the figures illustrate merely one possible implementation of the present invention. 
The picture signals 14 represent an image composed of a matrix of rows and columns of 
pixels. The red, green and blue picture signals 14, 16 comprise in a digital format information 
about the red, green and blue color components of the pixels of an image. In case of real-time 
video signals, these signals are handled as a sequence of images. To each of Ihese hnages the 
10 processing can be applied. 

In Fig. 2, there is illustrated, in somewhat greater detail, one possible 
embodiment of Fig. 1 and each of the processing blocks illustrated therein is now discussed 
in turn. 

A first converter block 22 saves to convert the Red R, Green G, Blue B iaput 
I5_picture.signals 1 4 t«tr^ RrigTttneoa V, Hne and Satoration-S. Parameter. values. . 

In the illustrated embodiment, this block can be split functionally in two sub- 
blocks, i.e. a linear transformation accomplished through a matrix followed by a non-linear 
block. 

Through the linear block Brightness Y and chromaticity coordinates Q, Cbare 
20 calculated for all pixels as follows: 



Y 




anai2ai3 




R 


Cr 




^21^22323 




G 


Cb 




a31%2*33_ 




B 



9 



wherein the matrix ay comprises a predetermined set of parameters, which depend on the 
25 color coordinates of the display device to be driven. This computation can be executed in 
hardware by using three multipliers and three adders for each row. 

The saturation S and hue values H for a pixel are then defined through the 
following formulae: 



30 



H==tan(C,/Cb). 
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These functions can be implemented in several ways for example by using 
Look Up Tables (LUTs) and/or hardware interpolators. 

A non-linear saturation processing block 24 is arranged to receive maximum 
saturation values Smax and the saturation values firom the converter 22. How the maximum 
saturation value Smax is detemiined for a pixel will be explained later. In the illustrated 
embodiment a non-linear function in the form of a gamma function is employed as follows: 

s'-s_(s/s_y\ 

wherein S' is the output parameter of flie ou^ut picture signals 16 corresponding to the 
saturation and y,, the power used in this non-linear processing. 

The transfer function can be rearranged in this form: 

- g»_ 2Y^togl(s)^Jdgi(S^)]+lo6,(s^) — 

This function can be implemented in several ways, for example using LUTs 
and multipliers. One possible embodiment is illustrated in Fig. 3. 

Firstly the logarithm of the saturation S and maximum saturation value is 
determined in req)ective logarithmic blocks 302, 301 . Next, fte results are subtracted in - 
subtracter 303. The output of the subtracter 303, being Log2 (S) - Log^ (S„^ ) , is multiplied 
with the power in multiplier 304. The output of the multiplier 304 is added to the output 
Logj (S^ ) of block 301 in adder 305. The output of &e adder 305 is used as power in 
exponential block 306, wherem the number two is raised to this power. The result, present at 
the output of the exponential block 306, is the output saturation value S* . 

A saturation bound evaluation block 34 receives both tiie hue values H from 
the converter 22 and the output brightness value Y' from the luminance-processing block 32. 

The graph of Fig. 5a illustrates a typical relationship between the maximxmi 
saturation values and the brightness values Y for a given Hue value H. Intermediate 
parameters , mj and Y^ are functions of the hue values H, These values can be readily 
obtained from a LUT. Inteimediate parameters mi and ma are defined as a tangent of the 
respective angles ai, and oz as shown in Fig. 5a; 
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mi=tg(ai) 
ma = tg (a2) 

5 In Fig. 5b, 5c, 5d smtable values for the respective intennediate parameters 

m m and Y» are provided. These suitable values have experimentally been found in a 
situation, -where the input parameter values are available m an eight bit digital format 
(resulting in 256 possible different values) and the matrix ajj contains following values: 

10 an = 66 au=129 ai3 = 25 

a.21 = -38 a22 = -74 aaa = 1 12 

asi = 112 a32 = -94 aaa = -18 

When applying this matrix a^, the minimum brightness value YtOn (as shown in 
-i5 i4g^^a)-is-eqoatto-zero, while the maximum-bri^i^^ 

A block diagram of a possible hardware implementation of the saturation 
bound evaluation block 34 is given in Fig. 6a. A first branch calculates the function 
(Y,nK - Y')-m2 by means of first subtractor 601 and first multiplier 602. A second branch 
calculates the fiinction (Y'-Y^ Vi ^"^8 second subtractor 603 and second multipUer 604. 
20 Comparator 605 determines whether Y' is larger or smaUer than Yt. If Y'- is larger tiian Y. 
multiplexer 606 selects (Y^ -Y'K as output. If Y* is smaller tiian Y. then multiplexor 
606 selects (Y"-Y^ )m, as output. In this way the output of the multiplexer 606 corresponds 
to the desired maximum saturation values Sm« as function of the output brightness values Y' 
as shown in Fig. 5a. 

25 The interaiediate parameters mi, m.2 and Yt are obtainable fix>m LUTs, which 

receive the hue value H as input (not shown in Fig. 6a). These LUTs contain the interaiediate 
parameters as shown in liie graphs of Figs. 5b, 5c, 5d. 

An alternative blockdiagram of a possible hardware implementation of the 
saturation bound evaluation block 34 is given in Fig. 6b. The comparator 610 detennines 
30 again whether Y' is larger than Yt: 

If no, the multiplexers 611, 612, 615 select as output the input values present 
at the input indicated by the dotted lines inside the multiplexers 61 1, 612, 615. 
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If yes, the multiplexers 61 1 , 612, 61 5 select the input values present at the 

other inputs. 

As a result, at the output of the subtractor 613 will be present (Y'-Y^ ), or 
respectively (Y^ -Y'). 

The multiplier 614 multiplies the mentioned outputs of subtractor 613 with mi, 
respectively m2 depending on the output of comparator 610. 

The multiplexer 615 selects mi, respectively ma depending on the output of 
comparator 610. The resulting output of the multiplier 614 is again the desired maximum 
saturation value Smax^ 

S„.ax-(Y-Y^K,ifY»<Y, 

S«ax=(Y^-Y0m2,ifY>Y, 

As mentioned before, the intermediate paramet^ are obtainable from LUTs. 

A y h LUT block 26 receives the hue values H from the converter 22. The 

value can comprise a function o f hue, hi this wav it is possible to applv d iffermt color 

saturation processing depending on the hue values H. This can be realized easily using a 
LUT. An example of suitable values of the power y,, as function of the Hue values H is 
given in Fig. 7. Hie power y^ , shows larger values (close to one) within a limited range of 
hue values H. This limited range includes, for example, the hue values H around 145 
corresponding to skin colors. The human eye is veiy sensitive to these colois and an increase 
of saturation would result in '^unnatural" images. By reducing the degree of non-linear 
processing (by selecting powers closer to one) for these skin colors, this problem is avoided. 
Outside the limited range a considerable "boosting^ of the colors is obtained by applying, for 
example, a value of 0.7 for y^ . 

Turning back to Fig. 2, a non-linear luminance processing block 32 is arranged 
to receive the brightness value Y from the converter 22. 

In this embodiment a non-linear function, using another power y y is applied 
wherein the output brightness values Y' is provided by the function: 

The transfer function can be rearranged in below form: 



■ * 
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Above function can, for ejcample, be realized as shovm in the hardware block 
diagram of Fig. 4. The logarithm of the input brightness value Y is determined in the 
5 logarithmic block 401 . The output of the logarithmic block 401 is multiplied with the other 
power Y Y in multiplier 402. The output of multiplier 402 is used as power m the exponential 
block 403 to raise the number two to this power. The result present at the output of the 
exponential block 403, is the ou^ut brightness value Y' . An alternative for above hardware 
block diagram is to apply LUTs. 

10 hi Fig. 2, a LUT Block 28 also receives the hue values H from ftie 

converter 22. The y y value can also comprise a function of hue, and so it is possible to apply 
different brightness processing depending on the hue values H. This can be easily realized by 
way of an appropriate LUT. An example of suitable values of the other power Yy as function 
of the hue values H is given in Fig. 8. The other power Yy shows a value close to one within 

15 a Umited range of the hue values H (around 145). Again, in this range the degree of non- 
linear processing is reduced (or made zero) in order to avoid unnatural skm colors. Outside 
this range, the other power Y y ^ preferably a value larger than one, for example 1 .2. 

In Fig. 2, a YHS to RGB converter 30 comprises the last block in the s^nal 
flow path and serves to convert the output parameter values Y' , S' , H to flie output picture 

20 "signals R'G'B'. 

This block can be implemented by cascading a non-linear transformation 
function for obtainir^otitputdiromaticity coordinates Cr' and Cb' from the ouQ)ut 
saturation values S' and hue valxies H, followed by a linear transformation converting these 
coordinates in output picture signals R' , G' and B' . 

25 The formulae for calculating output coordinates Cr' and Cb' are: 

Cr'=S'sm(H) Cb'=S'cos(H) 

The functions sin(H) and cos(H) can be implemented through Look Up Tables 
30 and interpolators. 

Further, the values of ou^ut picture signals R' , G' , B' , can be calculated by 

multiplying Y' , C, ' , C^ ' by an appropriate matrix by as shown below. 
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'R'' 




G' 




B' 





b3lb32b33 





'Y' ' 











wherein the matrix by comprises a predetermined set of parameters depending 
on the color coordinates of the display device. The function requires three multipliers and 
three adders for each matrix row. 

The non-linear processing can also be made dependent on the distribution of 
the input parameter values S, H of an image. This distribution can be evaluated by using a 
histogram evaluator 20 as shown in Fig. 2. The evaluator 20 receives the input parameter 
values Y, S, H and determines the number of occurrences of combinations (triplets) of input 
values Y, H, S in an area (a '^window'O of an image (or in the whole image). 

Depending on the distribution the powers y y and yj^ can be adapted in 
—resp ectively the yy LUT 28 -and'^^bUT 26 via micro-contr oller 18. As -airesQit-tiie-degree — 
of non-linear processing can be adapted depending on the distribution. Since the main goal is 
to adapt for a given hue value H the color saturation of an area of an image, a suitable 

parameter to use is the average saturation value S (Y)j ^ , which is a function of the input 
brightness value Y. 

This average saturation value S(Y)j ^ as function of input brightness values Y 
can be derived from the histogram Q ( Y, H, S) as follows: 



S(Y) 



H = 2 s,n|H(Y,s,), 



j 

wherein is the histogram for a given hue value H, i is an index for the range of possible 
input saturation values S' and S| is the i-th possible input saturation value. In case of 8 bits 
digital input picture signals, the range of i is 256. 

A refinement can be applied by taking into accoxmt a weighing factor 
W ( Y, H) , that weights tiie average saturation values S(Y)| ^ depending on the maximum 
possible saturation values for a given input brightness value Y. 

Taking into account the weighing fector W(Y JS), the weighed average 
saturation value becomes: 
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S'(Y)|h =W(Y,H)S(Y)|h = s^^^"h) 

As next step a parameter v|/| ^ , being a weighed averse hue saturation value 
for a given hue value, can be calculated from the expression: 



H=SYiS'(YplH, 



wherein j is an index for the range of possible input brightness values Y. 

The weighed average hue saturation value h related to an area of an image 
can be used to adapt the degree of non-linear processing by adapting the power . 

In case of video information with changing areas in an image or changing 
10 hnages, the value \j/|h is determined for each image of the video information. The current 

value v|/|h is compared with a user adjustable target value VtIh • 



If the current value « deviates from the target value I h » ^ ^^^^ 

signal, proportional to the difference of the 2 values is used to adapt the power • By 

applying a low pass filtermg on the error signal, abmpt changes of the power y^ due to 

15 changing video information are avoided. 

In this way disturbing effects are avoided, when the output picture signals are 

displayed on the display device. 

It should be noted that the above-mentioned embodiments illustrate rather than 
limit the invention, and that those skilled in the art will be able to design many alternative 

20 embodiments without departing from the scope of the appended claims. In the claims, any 
reference signs placed between parentheses shall not be constmed as limiting the claim. The 
word "comprising" does not exclude the presence of elements or steps other than those listed 
in a claim. The word "a" or "an" preceding an element does not exclude the presence of a 
plurality of such elements. The invention can be implemented by means of hardware 

25 comprising several distinct elements, and by means of a suitably programmed computer. In 
the device claim enumerating several means, several of these means can be embodied by one 
and the same item of hardware. The mere fact that certain measures are recited in mutually 
different dependent claims does not indicate that a combination of these measures cannot be 
used to advantage. 
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1 . A method of non«>linear processing of at least one set of iapvA parameter values 

of input picture signals so as to produce output picture signals with output parameter values, 
characterized in that the non-linear processiag is re^onsive to hue values of the input picture 
signals. 



2. A method as claimed in Claim 1, whwein the non-linear processing involves 
the steps of detemiining a power depending on the hue values, and raising a saturation- 
related input parameter value S to the power . 

3 . A method as claimed in Claim 2, further comprising the step of adapting the 
power based on histogram data derived from the input parameter values. 

4» A method as claimed in Claim 1, wherein the non-linear processing involves 

the steps of determining an power y y depending on the hue values, and raising a brightness- 
related input parameter value to the power y y . 

5. A method as claimed in Claim 4, further comprising the step of adapting the - 

power y y based on histogram data derived from the input parameter values. 

6- A method as claimed in Claim 2, wherein the non-linear processing of the 

saturation related input parameter value S depends on maximum saturation values Smax- 

7. A method as claimed in Claim 6, wherein the maximum saturation values Smax 

depend on the hue values. 

8- A method as claimed in Claim 6, wherein the maximum saturation values Smax 

depend on a brightness-related ou^ut parameter value. 
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9. A method as daimed in Claim 6, wherein a saturation-related output 

parameter value S' is substantially determined by the equation: 

s'=s^(s/s^y". 

5 10. A method as daimed in Claim 3, wherein for a predetermined hue value the 

power Yh is ad^ted based on a weighed, average saturation value of the input picture signals, 
representing pixels in a window of an image. 

11. A method as claimed in Claim 10, wherem for a predetermined hue value the 
10 power y h for a current window is adapted depending on the histogram data of the current 

and/or a previous window. 

12. An apparatus comprising picture processing circuitry for carrying out the 
method as claimed in claim 1. 

15 
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ABSTRACT: 



The present invention provides for a method of picture processing for a 
display device charactraized by the steps of processing input picture signals (R,G3) in a non- 
linear manner so as to produce output picture signals (R',G'3') for the device, such non-linear 
processing being responsive to parameters (Y,SJFI) of the input picture signals (R,G3). 



Fig.l 
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